<html>
<head><meta charset="utf-8"><title>Where are third party crates in rustc-src? · t-compiler · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/index.html">t-compiler</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Where.20are.20third.20party.20crates.20in.20rustc-src.3F.html">Where are third party crates in rustc-src?</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="213309921"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Where%20are%20third%20party%20crates%20in%20rustc-src%3F/near/213309921" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Where.20are.20third.20party.20crates.20in.20rustc-src.3F.html#213309921">(Oct 14 2020 at 15:47)</a>:</h4>
<p>Rust stdlib uses certain external crates internally: <a href="https://github.com/rust-lang/rust/blob/5565241f65cf402c3dbcb55dd492f172c473d4ce/library/std/src/os/mod.rs#L6">https://github.com/rust-lang/rust/blob/5565241f65cf402c3dbcb55dd492f172c473d4ce/library/std/src/os/mod.rs#L6</a></p>
<p>However, those crates seem to be missing from the rustc-src component:</p>
<div class="codehilite"><pre><span></span><code>λ exa -l (rustc --print sysroot)/lib/rustlib/src/rust/library
drwxr-xr-x - matklad 11 Oct 18:21 alloc
drwxr-xr-x - matklad 11 Oct 18:21 backtrace
drwxr-xr-x - matklad 11 Oct 18:21 core
drwxr-xr-x - matklad 11 Oct 18:21 panic_abort
drwxr-xr-x - matklad 11 Oct 18:21 panic_unwind
drwxr-xr-x - matklad 11 Oct 18:21 proc_macro
drwxr-xr-x - matklad 11 Oct 18:21 profiler_builtins
drwxr-xr-x - matklad 11 Oct 18:21 rtstartup
drwxr-xr-x - matklad 11 Oct 18:21 rustc-std-workspace-alloc
drwxr-xr-x - matklad 11 Oct 18:21 rustc-std-workspace-core
drwxr-xr-x - matklad 11 Oct 18:21 rustc-std-workspace-std
drwxr-xr-x - matklad 11 Oct 18:21 std
drwxr-xr-x - matklad 11 Oct 18:21 stdarch
drwxr-xr-x - matklad 11 Oct 18:21 term
drwxr-xr-x - matklad 11 Oct 18:21 test
drwxr-xr-x - matklad 11 Oct 18:21 unwind
</code></pre></div>


<p>Are they present anywhere else?</p>



<a name="213309986"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Where%20are%20third%20party%20crates%20in%20rustc-src%3F/near/213309986" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Where.20are.20third.20party.20crates.20in.20rustc-src.3F.html#213309986">(Oct 14 2020 at 15:47)</a>:</h4>
<p>IDEs would need those to analyzer everything properly...</p>



<a name="213310028"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Where%20are%20third%20party%20crates%20in%20rustc-src%3F/near/213310028" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Where.20are.20third.20party.20crates.20in.20rustc-src.3F.html#213310028">(Oct 14 2020 at 15:47)</a>:</h4>
<p>cc <span class="user-mention" data-user-id="211727">@Jonas Schievink</span> , I think you've looked a bit into this</p>



<a name="213310113"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Where%20are%20third%20party%20crates%20in%20rustc-src%3F/near/213310113" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jonas Schievink  [he/him] <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Where.20are.20third.20party.20crates.20in.20rustc-src.3F.html#213310113">(Oct 14 2020 at 15:48)</a>:</h4>
<p>Yeah, this part looks intentional to me – we might not want to vendor every dependency</p>



<a name="213310164"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Where%20are%20third%20party%20crates%20in%20rustc-src%3F/near/213310164" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jonas Schievink  [he/him] <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Where.20are.20third.20party.20crates.20in.20rustc-src.3F.html#213310164">(Oct 14 2020 at 15:48)</a>:</h4>
<p>They're already locked in by Cargo.lock</p>



<a name="213310380"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Where%20are%20third%20party%20crates%20in%20rustc-src%3F/near/213310380" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Where.20are.20third.20party.20crates.20in.20rustc-src.3F.html#213310380">(Oct 14 2020 at 15:50)</a>:</h4>
<p>Hm.... Seems more logical to bundle everything into rust-src -- I don't think there should be fundamental differnece between library crates in rust-lang/rust and library crates elsewhere</p>



<a name="213310434"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Where%20are%20third%20party%20crates%20in%20rustc-src%3F/near/213310434" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jonas Schievink  [he/him] <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Where.20are.20third.20party.20crates.20in.20rustc-src.3F.html#213310434">(Oct 14 2020 at 15:50)</a>:</h4>
<p>Maybe, yeah</p>



<a name="213310482"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Where%20are%20third%20party%20crates%20in%20rustc-src%3F/near/213310482" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jonas Schievink  [he/him] <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Where.20are.20third.20party.20crates.20in.20rustc-src.3F.html#213310482">(Oct 14 2020 at 15:50)</a>:</h4>
<p>Would be nice not to have to do network requests to download them</p>



<a name="213310497"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Where%20are%20third%20party%20crates%20in%20rustc-src%3F/near/213310497" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Where.20are.20third.20party.20crates.20in.20rustc-src.3F.html#213310497">(Oct 14 2020 at 15:50)</a>:</h4>
<p>Yeah...</p>



<a name="213310567"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Where%20are%20third%20party%20crates%20in%20rustc-src%3F/near/213310567" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jonas Schievink  [he/him] <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Where.20are.20third.20party.20crates.20in.20rustc-src.3F.html#213310567">(Oct 14 2020 at 15:51)</a>:</h4>
<p>Might be more of a T-infra question though</p>



<a name="213310601"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Where%20are%20third%20party%20crates%20in%20rustc-src%3F/near/213310601" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Where.20are.20third.20party.20crates.20in.20rustc-src.3F.html#213310601">(Oct 14 2020 at 15:51)</a>:</h4>
<p>OTOH, running <code>cargo metadat</code> from withing rust-analyzer would be organizationally less painful than patching dist</p>



<a name="213310777"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Where%20are%20third%20party%20crates%20in%20rustc-src%3F/near/213310777" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Where.20are.20third.20party.20crates.20in.20rustc-src.3F.html#213310777">(Oct 14 2020 at 15:52)</a>:</h4>
<p>But that'd require <code>RUSTC_BOOTSTRAP</code>, as ws crates might use nightly cargo features</p>



<a name="213310831"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Where%20are%20third%20party%20crates%20in%20rustc-src%3F/near/213310831" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jonas Schievink  [he/him] <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Where.20are.20third.20party.20crates.20in.20rustc-src.3F.html#213310831">(Oct 14 2020 at 15:52)</a>:</h4>
<p>even for <code>cargo metadata</code>?</p>



<a name="213310984"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Where%20are%20third%20party%20crates%20in%20rustc-src%3F/near/213310984" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Where.20are.20third.20party.20crates.20in.20rustc-src.3F.html#213310984">(Oct 14 2020 at 15:54)</a>:</h4>
<p>Hm, maybe not? Quick testing works, but I remember this not working a while ago...</p>



<a name="213311091"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Where%20are%20third%20party%20crates%20in%20rustc-src%3F/near/213311091" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Where.20are.20third.20party.20crates.20in.20rustc-src.3F.html#213311091">(Oct 14 2020 at 15:54)</a>:</h4>
<p>THis also seems like another argument in favor of moving std to a separate workspace.</p>



<a name="213311120"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Where%20are%20third%20party%20crates%20in%20rustc-src%3F/near/213311120" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Where.20are.20third.20party.20crates.20in.20rustc-src.3F.html#213311120">(Oct 14 2020 at 15:54)</a>:</h4>
<p>cc <span class="user-mention" data-user-id="116122">@simulacrum</span></p>



<a name="213311482"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Where%20are%20third%20party%20crates%20in%20rustc-src%3F/near/213311482" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Where.20are.20third.20party.20crates.20in.20rustc-src.3F.html#213311482">(Oct 14 2020 at 15:56)</a>:</h4>
<p>There's a rust-src component which vendors everything</p>



<a name="213311567"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Where%20are%20third%20party%20crates%20in%20rustc-src%3F/near/213311567" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Where.20are.20third.20party.20crates.20in.20rustc-src.3F.html#213311567">(Oct 14 2020 at 15:57)</a>:</h4>
<p>We can vendor just std deps but iirc there's no easy way to subset a cargo.lock</p>



<a name="213311592"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Where%20are%20third%20party%20crates%20in%20rustc-src%3F/near/213311592" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Where.20are.20third.20party.20crates.20in.20rustc-src.3F.html#213311592">(Oct 14 2020 at 15:57)</a>:</h4>
<p>Or I guess subgraph</p>



<a name="213311608"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Where%20are%20third%20party%20crates%20in%20rustc-src%3F/near/213311608" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Where.20are.20third.20party.20crates.20in.20rustc-src.3F.html#213311608">(Oct 14 2020 at 15:58)</a>:</h4>
<p>Cc <span class="user-mention" data-user-id="120518">@Eric Huss</span></p>



<a name="213313448"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Where%20are%20third%20party%20crates%20in%20rustc-src%3F/near/213313448" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Where.20are.20third.20party.20crates.20in.20rustc-src.3F.html#213313448">(Oct 14 2020 at 16:09)</a>:</h4>
<p>Hm, <code>rust-src</code> is what I am talking about, and it doesn't have 3rd party crates</p>



<a name="213313555"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Where%20are%20third%20party%20crates%20in%20rustc-src%3F/near/213313555" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Where.20are.20third.20party.20crates.20in.20rustc-src.3F.html#213313555">(Oct 14 2020 at 16:10)</a>:</h4>
<div class="codehilite"><pre><span></span><code>18:09:32|~/projects/rust-analyzer|master✓
λ rustup component add rustc-dev
info: component &#39;rustc-dev&#39; for target &#39;x86_64-unknown-linux-gnu&#39; is up to date

18:09:41|~/projects/rust-analyzer|master✓
λ rustup component add rust-src
info: component &#39;rust-src&#39; is up to date

18:09:47|~/projects/rust-analyzer|master✓
λ fd -e toml . (rustc --print sysroot) | rg cfg
</code></pre></div>



<a name="213313645"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Where%20are%20third%20party%20crates%20in%20rustc-src%3F/near/213313645" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jonas Schievink  [he/him] <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Where.20are.20third.20party.20crates.20in.20rustc-src.3F.html#213313645">(Oct 14 2020 at 16:10)</a>:</h4>
<p>I think you're talking about different things (locking dependency versions in the lockfile vs. copying the sources of all dependencies into the rust-src component)</p>



<a name="213313984"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Where%20are%20third%20party%20crates%20in%20rustc-src%3F/near/213313984" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Eric Huss <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Where.20are.20third.20party.20crates.20in.20rustc-src.3F.html#213313984">(Oct 14 2020 at 16:12)</a>:</h4>
<p>Yea, I don't think <code>cargo vendor</code> can do just a partial vendor.  <code>bootstrap</code> would need to collect the dependencies.  How much larger would that make the rust-src component?  I would be concerned about it getting too bloated, but I suspect it's not too much.  I don't know anything about ra, but can it use whatever mechanism it uses for normal user dependencies to access the source? Or does it need rmeta files or something?</p>



<a name="213317459"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Where%20are%20third%20party%20crates%20in%20rustc-src%3F/near/213317459" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Where.20are.20third.20party.20crates.20in.20rustc-src.3F.html#213317459">(Oct 14 2020 at 16:37)</a>:</h4>
<p><span class="user-mention" data-user-id="120518">@Eric Huss</span> if there are <code>Cargo.toml</code> and <code>Cargo.lock</code> in <code>rust-src</code> and stable <code>cargo metadata</code> would work for those, that'll work</p>



<a name="213317681"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Where%20are%20third%20party%20crates%20in%20rustc-src%3F/near/213317681" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Where.20are.20third.20party.20crates.20in.20rustc-src.3F.html#213317681">(Oct 14 2020 at 16:38)</a>:</h4>
<p>The drawbacks would be:</p>
<ul>
<li>internet access required to get full completions for std</li>
<li>we'd have to manually prune the <code>cargo metadata</code> graph to separate <code>library</code> from <code>compiler</code></li>
<li>no super-bullet-proof guarantee that we use the same sources that were used to compile the library</li>
</ul>



<a name="213318171"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Where%20are%20third%20party%20crates%20in%20rustc-src%3F/near/213318171" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Eric Huss <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Where.20are.20third.20party.20crates.20in.20rustc-src.3F.html#213318171">(Oct 14 2020 at 16:42)</a>:</h4>
<p>I see.  We've talked about making a synthetic <code>Cargo.toml</code> (<a href="https://github.com/rust-lang/wg-cargo-std-aware/issues/27">https://github.com/rust-lang/wg-cargo-std-aware/issues/27</a>), but it might be tricky.</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>